Digital images and video come in a large variety of formats. For many applications, converting between two or more different formats is desirable. A high-quality low-cost method for converting the digital signals is very useful for such applications as: (1) converting interlaced NTSC video at 60 fields/second to progressive video with a similar or larger horizontal and vertical resolution at 60 frames/second for display on progressive televisions, (2) performing a high-quality “zoom” function on either interlaced or progressive video and (3) increasing the horizontal and/or vertical resolution of progressive or interlaced video or images.
Existing solutions for video deinterlacing include bob (i.e., vertical spatial filter), weave (i.e., temporal filter), VT-filter (i.e., vertical spatial filter combined with temporal filter, commonly fixed filtering that combines a highpass version of a previous opposite parity field with a lowpass interpolation of a missing line from a current field), motion-adaptive and motion-compensated techniques and edge-based spatial filtering. The various video techniques that are not temporal in nature are applicable to image up-conversion (i.e., vertical and edge-based spatial filtering). Horizontal and edge-based spatial filtering is used for horizontal upsampling of images or video.
Bob (i.e., vertical filtering) is known to produce temporal flickering artifacts in video and reduced vertical detail in both images and video. In vertical filtering, odd and even lines are alternately blurred in the video by interpolation in a vertical direction only from adjacent lines. A resulting lack of vertical detail is particularly noticeable for sharp edges. Weave (i.e., temporal filter) is known to produce “jaggies” (i.e., interlace artifacts that are extremely objectionable for moving objects). The VT-filtering is a fixed (i.e., non-adaptive) low-cost line-based process that is cost effective to implement in silicon but is known to produce temporal artifacts (i.e., trailing edges or “edge ghosts” from previous fields appear behind moving objects).
Motion adaptive techniques commonly make pixel-level, block-level and/or picture-level decisions about whether to use weave or bob or a blended combination of weave and bob for particular pixels, blocks and/or pictures. Weave is a good option for still portions of video and a poor choice for moving areas. Hard block-level decisions in motion adaptive techniques can lead to objectionable blocking artifacts. However, more advanced motion adaptive deinterlacing techniques that combine weave and bob suffer mainly from relatively poor performance for moving video due to all the drawbacks of bob. For stationary regions, however, the flickering artifact created by bob may be reduced.
Motion compensated techniques operate in a similar manner to motion adaptive techniques, except that rather than always using co-located pixels from a previous opposite parity field to replace missing pixels in a progressive frame that is formed from the current field (i.e., weave), motion compensated pixels are chosen from the previous opposite parity field. An advantage of the motion compensated technique is that good deinterlacing is achievable for moving video that can be well estimated. A disadvantage of the motion compensated technique is that motion estimation is often more expensive than any of the previously mentioned techniques. If motion estimation fails on the video sequence (i.e., highly irregular motion, non-smooth motion fields or various lighting effects), motion compensated techniques may be no better than less complex methods. Furthermore, even when motion estimation is successful, an amount of high-frequency information that can be transferred from the previous opposite parity field to the estimate of the missing lines for reconstruction a progressive frame from the current field depends upon a sub-pel motion between the two fields. In a worst case, objects can move by an integer number of pels plus exactly one-half pel in the vertical direction in the temporal interval between the previous field and current field. Therefore, no additional high-frequency vertical information for the missing lines of the current field is gleaned from the previous field through the motion compensated estimate. In practice, however, motion compensated deinterlacing increases vertical detail while reducing flickering artifacts on a broad range of video, such that a common drawback is simply complexity.
Edge-based spatial filtering operates on only the current field and is capable of producing a better estimate of the pixels from the missing lines than what is possible with vertical filtering only. To a lesser extent than bob, edge-based spatial filtering also suffers from lack of vertical detail. In particular, high frequency textures that lack edges will not be improved over simple bob.